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The National Aeronautics and Space Administration (NASA) is embarking on a new era of Space Exploration, aimed at 
sending crewed spacecraft beyond Low Earth Orbit (LEO), in medium and long duration missions to the Lunar surface. Mars 
and beyond. The challenges of such missions are significant and will require new technologies and paradigms in vehicle 
design and mission operations. Current roles and responsibilities of spacecraft systems, crew and the flight, control team, for 
example, may not be sustainable when real-time support is not assured due to distance-induced communication lags, radio 
blackouts, equipment failures, or other unexpected factors. Therefore, technologies and applications that enable greater 
Systems and Mission Management capabilities on-board the space-based system will be necessary to reduce the dependency 
on real-time critical Earth-based support. The focus of this paper is in such technologies that will be required to bring 
advance Systems and Mission Management capabilities to space-based environments where the crew will be required to 
manage both the systems performance and mission execution without dependence on the ground. We refer to this concept as 
“autonomy.” Environments that require high levels of autonomy include the cockpits of future spacecraft such as the Mars 
Exploration Vehicle, and space-based control centers such as a Lunar Base Command and Control Center. Furthermore, this 
paper will evaluate the requirements, available technology, and roadmap to enable full operational implementation of on- 
board System Health Management, Mission Planning/re-planning, Autonomous Task/Command Execution, and Human 
Computer Interface applications. The technology topics covered by the paper include enabling technology to perform 
Intelligent Caution and Warning, where the systems provides directly actionable data for human understanding and response 
to failures, task automation applications that automate nominal and off-nominal task execution based on human input or 
integrated health state-derived conditions. Shifting from Systems to Mission Management functions, we discuss the role of 
automated planning applications (tactical planning) pn-board, which receive data from the other cockpit automation systems 
and evaluate the mission plan against the dynamic systems and mission states and events, to provide the crew with 
capabilities that enable them to understand, change, and manage the timeline of their mission. Lastly, we discuss the role of 
advanced human interface technologies that organize and provide the system and mission information to the crew in ways 
that maximize their situational awareness and ability to provide oversight and control of all the automated data and functions. 


1. Introduction 

NASA and other space agencies, have renewed goals of 
expanding our domain of space travel and exploration, 
anchored in LEO for the last 3 decades. Propelled by a new 
mandate to return to the Moon in the next decade and to 1 
eventually send humans to Mars, NASA has initiated the 
execution of the early phases of that mandate. In the near 
term, a new crewed spacecraft will be developed to replace 
the Space Shuttle as the main US-provided transportation 
system to LEO, and eventually transport astronauts to lunar 
orbit. However subsequent space exploration missions 
beyond LEO will face challenges comparable to those 


experienced by the engineers and scientists that planned, 
designed, developed and executed the first space pioneering 
missions, such as the early Russian and American orbital 
flights, and ultimately the Apollo program. The new space 
exploration missions will combine two characteristics that 
make them especially challenging for human spaceflight: 
long distance and long duration. Furthermore, these 
missions will require system capabilities that have not yet 
been designed or applied to human spaceflight, and may 
lead to paradigm shifts in the way we design for or execute 
such human space missions. One such requirement is the 
migration of a much greater share of the management of the 
vehicle systems and mission to the space-bound or space- 


1 


based crew. This is due primarily to the intrinsic 
communication lags, and necessity of the crew to take 
control of their systems if the connection with the Mission 
Control Center is lost. With these challenges in mind, this 
paper explores the capability requirements, and 
technologies to be considered in the design of systems for 
future cockpits or space-based control centers to meet and 
mitigate the challenges to next-generation human space 
exploration. 

Technology Needs vs. Technology Solutions 

The discussion of new capability needs for future spacecraft 
systems is often approached from two different angles, 
depending on the source of the discussion. The research 
community, for example, has invested significant effort in 
developing and maturing new technologies that may 
provide innovative capabilities on-board the spacecraft if 
implemented (e.g. Integrated Vehicle Health Management 
(IVHM) or spacecraft automation). From the operations 
perspective, however, discussion on specific technology 
takes a second role to required improvements on specific 
functions to meet new program and missions’ goals, 
objectives, and ultimately program requirements (e.g. 
maintaining crew safety despite limited communications 
coverage). In this paper, we attempt to outline the Systems 
or Mission Management requirements for long-duration, 
long-distance human space missions, and evaluate 
technologies or applications that may provide the required 
capabilities, 

2. Systems/Mission Management 
Evolution: from LEO to Mars 

Space systems required to support long distance / extended 
duration missions beyond LEO will have a complexity 
beyond what we have experienced in any previous human 
space program. Beyond the basic technical challenges 
derived from the distance or duration characteristics, such 
as communication infrastructure, propulsion systems, etc, 
there are other factors that further complicate mission 
operations. The logistics approach to support the execution 
of such mission will far exceed the complexity of any 
previous program, and most likely will have significant 
impact on the system design - simply put, there will be no 
FedEx or UPS to deliver spare parts to the far reaches of the 
solar system! 

During the Apollo program, while the. missions included 
Moon-surface operations, the duration of the longest 
mission was 12 days; therefore issues such as the storage of 
consumables, system maintenance, and logistical sparing 
were not a significant concern. Also, the location of the 
surface operations (near side of the moon) along with the 
duration of such, limited the amount of communication 
blackouts from Earth. Skylab was the first NASA program 
to experience long-duration mission characteristics; 


however the mission duration for the longest crewed visit 
was only 84 days. Due to the multiple technical issues 
experienced during the launch (early solar shield shroud 
deployment and subsequent complete damage to solar array) 
the flight operations team had to perform extensive analysis, 
automated control of the vehicle, and preparation of 
numerous repair tasks (via EVAs) to sustain extended 
manned missions. Although the three Skylab crews were 
successful in completing several science experiments, much 
of their time was dedicated to system maintenance. Today’s 
Space Shuttle missions are in LEO for a typical mission 
duration of 1-2 weeks. Systems maintenance during a 
mission is usually not an issue and is limited to 
malfunctions that affect safety-critical functions. In most 
cases, critical functionality is preserved through system 
redundancy engineered into the vehicle design. Due to the 
relatively short duration of Shuttle missions, logistics is also 
not an issue, since the system is self-contained and no 
sparing is usually necessary. The International Space 
Station (ISS) is arguably the most complex space platform 
ever built, combining systems designed and built in several 
countries, hosting multiple types of visiting vehicles, and 
comprised of several laboratory modules. In many cases 
these different multi-national components will not be mated 
or electronically connected until their final assembly on- 
orbit. Even in its current incomplete assembly state of 5 
habitable modules, 3 docking ports, 2 airlocks, and 3 truss 
components, the ISS has undergone extensive system 
maintenance, both internal and external (via EVAs). These 
maintenance periods have included repairs and replacement 
of electrical switch boxes, air generation systems, command 
and control computers and attitude control Gyroscopes. 
Unlike the Space Shuttle, most sensor data and control 
co mm ands are available electronically and can be viewed 
and controlled from on-board laptops and via command 
uplinks and telemetry, with over a hundred thousand 
channels of data. - 

Considering this level of complexity and the limitations of 
on-board data storage and processing, it is reasonable to 
expect that most real-time andoff-line systems management 
and mission execution management for the ISS is performed 
by or with extensive support from the flight control team on 
Earth. In all cases, the ultimate interface between system 
“raw” data, mission timeline / flight plan, and mission goals 
and objectives is the human, both on-board and on the 
ground. Moreover, the level of automated processing and 
integration of system and mission data has increased 
significantly across all the programs discussed. Yet, in most 
cases it stifi .requires significant human processing to reach 
a sufficient level of understanding of vehicle systems or 
mission level significance. 

In the case of systems management, for example, to obtain a 
system-level health state assessment, subsystem data often 
requires additional diagnosis, correlation with other system 
data, and evaluation of its impact to other subsystems or 
functional availability. Furthermore, if system-level actions 
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(multi-subsystem commands) are required for nominal or 
off-nominal events, humans are required to trigger or 
manually perform them based on their assessment of the 
system-level health, mission phase, etc. In mission 
management, an example is the impact of system events to 
the mission goals or timeline. The humans integrate the 
individual subsystem and system-level health state 
assessment, with timeline requirements, such as required 
systems to perform an activity, and mission goals. Based on 
the dynamic parameters of such, they can execute as 
planned, re-plan, or potentially modify mission goals and 
objectives, depending on remaining functional availability 
and resources. Obviously, such human-centered 
management requires extensive system and mission 
knowledge, along with complex training and 
multidisciplinary human interaction, such as crew-to- 
Mission Control Center (MCC), or controller-to-controller 
interface in the MCC. Furthermore, it requires multiple 
groups of people performing different functions, from 
system analysis to planning. Whereas the on-board crews 
retain the final decision authority, and are the designated 
“first-responders” for conditions that require time-critical 
actions, they remain dependant on MCC support for real- 
time tactical support, as well as strategic support. 

Historically, this operations paradigm has held true and 
effective for the last 40 years of spaceflight. However, the 
characteristics of future Space Exploration missions may 
challenge this paradigm through several factors, including 
degradation of communications effectiveness, logistics 
complexity, safe-haven/ emergency retum-to-Earth options, 
and the number of manned / unmanned systems to manage 
and control. The complexity of the space systems to 
perform these missions is probably going to be equal to or 
higher than the most complex systems currently operated. 
At the same time, the availability of near-real-time support 
of MCC for their traditional tactical Systems and Mission 
management will decrease, in some cases significantly. 
Also, the requirements to maintain critical functionality to 
keep the crew alive and provide “Safe-Haven” will be 
substantially higher than in cuirent systems, since the 
retum-to-Earth options may be on the order of weeks or 
even months. Beyond the technical feasibility, it may not 
be fiscally viable to have the same level of ground support 
we currently have for ISS or Shuttle, on several systems that 
may potentially be needed at the same time (e.g. ISS, CEV, 
Lunar Base, Mars Exploration Vehicle, etc.). 

Therefore, future spacecraft systems must accommodate 
system capabilities that enable the transition (as depicted in 
Figure 1) from human-centered real-time Systems and 
Mission Management, to human oversight of real-time 
management performed by the systems in an automated or 
autonomous fashion. 

This transition should result in cockpit or space-based 
Control Center capabilities that enable the crew to obtain 
the necessary processed information of their system state 


and its relation to the mission, much like they currently get 
from MCC. On-board systems should provide the following 
Systems and Mission Management capabilities: 

@ System health state determination, which correlates 
individual subsystem or element health and status 
information with inter-system dependencies, vehicle- 
level fault propagation, and other relevant mission 
information, to provide the system-level health state. 

• Task/Procedure automation capabilities that execute 
system-wide nominal or off-nominal tasks depending 
on pre-defined conditions, system states or failures, and 
human-driven commands (and monitor/confimi the 
successful execution of such task sequences). This 
includes off-nominal Systems reconfiguration in 
response to failures and other anomalies that degrade 
spacecraft function or interfere with the execution of 
planned events. 

® Timeline planning and re-planning capabilities with 
the necessary built-in resource analysis tools linked to 
system data, to support on-board real-time automated or 
autonomous re-planning based on dynamic system 
availability, crew preference, or other mission 
conditions. 

® Human-Computer Interfaces with information 
management applications combined with advanced 
displays and vision systems that allow the crew to 
access and understand all the necessary data generated 
by the on-board systems, retaining the capability to 
provide efficient oversight and control of manual, 
automated or autonomous functions, while maximizing 
their situational awareness and minimizing cognitive 
overload during critical maneuvers or off-nominal 
events. 

Nevertheless, crews will require and continue to depend on 
strategic MCC support, including extended systems 
diagnosis, Mure troubleshooting, health -prognosis, 
maintenance, and mission planning. However, the 
effectiveness of such support to maintain crew safety and 
system availability to perform the mission is currently 
dependant on constant reliable communications. Moreover, 
other critical capabilities not discussed in this paper include 
on-board training, to maintain and enhance critical skills on 
long duration missions, and space-based maintenance 
support including the use of Class V Interactive Electronic 
Technical Manuals. (IETMs) linked to the system data for 
real-time maintenance procedure execution. 
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Figure 1: Evolution from human-centered direct systems and mission management, to human oversight of real-time 
autonomous management 


3. Transferring control on-board: The 
Crew Perspective 

Throughout the history of human space exploration, 
crewmembers have continually pushed for greater 
autonomy. From the earliest days of NASA’s Mercury 
program, the souls on board space vehicles have desired 
more insight as to their vehicle’s state, and options of 
control within their means onboard. Portholes / windows 
for viewing were an issue from the first capsule designs, 
through the International Space Station (ISS). A similar 
issue for a crew removable hatch has likewise been 
prevalent throughout the program. This need was 
magnified by the loss of three astronauts to a cockpit fire, 
during a pad practice countdown for Apollo L 

Reliance upon the crew’s ability to operate on their own has 
also shown through. From the lessons learned during 
manual proximity operations during the Gemini program’s 
initial rendezvous missions, the Apollo 11 descent and 
landing on the moon, the manual bums required during the 
Apollo 13 mission, and finally on to manual dockings and 
landings used throughout the shuttle program, manual crew 
inputs have been a mainstay of America’s human 
spaceflight program. Much effort and technology has been 
expended to enhance the information available to the 
crewmembers for which to base their inputs on. 


There is probably no better example of trying to put the 
right information in the hands of the crew to improve their 
situational awareness and autonomy than the proposed 
Shuttle Orbiter Cockpit Avionics Upgrade (CAU) to the 
Multifunction Electronic Display System (MEDS). When 
initially designed and implemented, the MEDS computer 
generated graphical displays were to mimic the original 
“steam gauge” mechanisms of the shuttle’s instrument 
panel. This was carried out to the extent of incorporating 
all the on-orbit attitude information on the Attitude 
Direction Indicator {ADI) during ascent and entry phases 
when they would be more of a distraction than pertinent. 
The mimicry of the ADI extended to the point of including 
the physical slot (belly band), which is an artifact of the 
hardware mechanism to mount the original ball within the 
instrument. 

The true benefits and capabilities of MEDS were promised 
as a future upgrade, which was eventually presented to the 
NASA astronaut office in the form of CAU. This upgrade 
made use of the MEDS color graphics displays, flexibility, 
and capabilities, and promised a major improvement to the 
crews’ situational awareness, and hence autonomy. It was 
user friendly in comparison to the heritage displays, and 
would promise a segue to Enhanced Caution & Warning 
(ECW); The decision to curtail the lifespan of the Shuttle 
Program, along with the loss of Columbia and ensuing 
decrease of remaining flights (along with budget concerns). 
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led to the cancellation of CAU just prior to implementation. 
Although the new Crew Exploration Vehicle (CEV) will 
lack the sophistication and capabilities of the Shuttle, the 
astronaut office will no doubt expect CAU to be a base- 
level starting point for avionics and display capabilities. 

NASA has spent decades in Low Earth Orbit (LEO). The 
administration is ready to move out of the LEO business 
and on to deeper space. In addition to bringing new 
excitement to the program, this also brings new challenges, 
such as communications, as explained in previous sections. 

This beckons for onboard Systems and Mission 
Management capabilities to assist the crew in monitoring 
systems, and guide them through procedures in the event of 
failure. An effective implementation needs to be flexible. 
Depending on the phase of flight, systems management 
capabilities can: direct the crew’s attention, diagnose the 
failure and prescribe a malfunction procedure, lead the crew 
through an imbedded procedure, or perform some or all of 
the procedure, informing the crew of its progress. 
Furthermore, it can be tailored to the needs / desires of each 
crew (just as some crews decided to train with 2 of 3 
displays set to Backup Flight Software (BFS) during ascent, 
with the original Shuttle cockpit). 

One of the most important aspects of the onboard systems 
will be to earn the confidence of the crew. This will require 
Systems and Mission Management automation capabilities 
developed with crew office input from day one. The system 
will need to be deployed early on to increase crew 
confidence, A bolt-on design will not exhibit the robustness 
necessaiy for the requirements of a flight to Mars. To prove 
worthwhile and successful, automation capabilities must be 
incorporated into the design of the entire system, from 
subsystems to multiple system elements. By having the 
system in place from the onset of the program, failure 
scenarios/signatures may be included in the knowledge base 
and compared with telemetry-based analysis within the 
MCC. This period of crosscheck and growth will inspire 
confidence in such capabilities and the technology that 
enable them, within the flight control team, the program 
office, and most importantly, within the astronaut corps. 
Last but not least, this build-up approach the 
implementation of technology that facilitates Systems and 
Mission Management automation decreases both fiscal and 
schedule risk; thus making it more palatable to the funding 
organizations for the program. 

4. Key Elements of a Smart Cockpit 

This section defines the specific functions and basic 
requirements for the Systems and Mission Management 
components that the authors propose for future spacecraft 
cockpits and space-based control centers. 


As illustrated in Figure 2, the management and autonomy 
functions in current manned spacecraft systems is often 
distributed amongst different components performing 
management and autonomy functions at different levels. It 
is important to understand the similarities and differences 
between these levels, since this paper targets the needs and 
potential solutions on some specific levels of systems and 
mission management. 

1. Subsystem Component level management. Subsystem 
hardware (not shown on Figure 2), including 
instrumentation, and firmware (FW) local to different 
subsystem components, which acquire sensor data and 
perform subsystem component-level processing, and 
Failure Detection, Isolation and Recovery (FDIR) locally. 
E.g , Electrical power switch sensing current and 
“tripping” (open-circuit) upon sensing the current above 
the designated upper-limit threshold. Sensor data 
validation is often handled at this level. 

2. Subsystem level management. Subsystem software 
(SW), which controls subsystem-specific functions, such 
as health and performance algorithms specified in 
different modes and states, as well as subsystem-level 
fault detection isolation and recovery (FDIR). The 
subsystem-level health management may include complex 
state estimation algorithms, limited parameter trending 
and FDIR actions bounded to that subsystem. E.g. 
Thermal Control System (TCS) software that detects a 
cooling pump malfunction (pump is off due to switch 
trip), and as response , sends commands to activate the 
backup pump and re-configures valves. 

3. Vehicle level management. Provides coordination and 
arbitration of subsystem performance, mission phase and 
vehicle mode and states transitions, and health 
management functions, such as FDIR that requires 
multiple subsystem inputs for detection and issues multi- 
subsystem commands as response. Command and 
Control (C&C) SW, and Enhanced C&W message 
generation are included in this jevei^ human- 
driven integration of subsystem data for vehicle-level 
health determination and management, or to integrate 
System Management data with Mission Management. 
E.g. C&W messages generated for electrical switch trip , 
and TCS pump off condition (no power). Operators 
identifying switch trip as “root-cause” failure, and 
manually optimizing system reconfiguration considering 
tripped switch, primary TCS pump un-powered 
Ultimately , operators (mostly) and crew evaluate impact 
of the failed switch and its loads (including TCS pump) to 
the system , and troubleshoot switch failure to determine if 
repairs or replacement are necessary. 

4. Mission level Management The functions on this level 
are related at integrating the performance of the vehicle, 
including health state and system availability, with 
mission goals, objectives, mission plan, etc. This is 
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currently a human-centered function, mainly directed and 
executed from MCC. E.g. Operators evaluate mission 
impacts derived from failed switch , and loss of power to 
its loads , and re-plan timeline to exclude activities with 
constrains associated with such switch or its loads . 



Figure 2 Distribution of Mission and Systems Management functions between subsystem components and SW, on-board 
autonomy capabilities, the crew and the Earth-based MCC. On this notional architecture, the on-board systems provide the 
crew with the necessary information and automation capabilities to manage the system and mission on-board. 


5. System level Management (not shown on Figure 2). 
Similar to vehicle-level but the coordination and 
arbitration, as well as health management integration 
occurring across multiple systems, such as multiple 
Lunar-base modules, or systems such as a Mars transfer 
vehicle-Mars Lander vehicle. 

Whereas technology evolution in all Systems / Mission 
management levels will be necessary to endeavor into the 
future space exploration missions, from more efficient 
hardware to advanced prognostic algorithms, this paper 
focuses on applications that provide automation capabilities 
at the Vehicle / System and Mission management levels, 
since such capabilities are in the critical path to meet the 
requirements of Systems and Mission Management 
automation. 

Vehicle / System Health Management 

The base, onto which critical automation capabilities for 
future cockpits or space-based control centers can be built 
upon, is the understanding of the vehicle or system health 
state. System Health Management starts at the 
instrumentation and Subsystem hardware levels, and is 
carried through the Vehicle and System management levels. 
Understanding the health of the system or vehicle is often 
the minimum requirement to execute other system and 


mission management functions, such as nominal scheduled 
tasks (e.g. EVAs, dockings, vehicle state transitions, etc,), 
or mission planning and execution. This fact drives the 
current manned spaceflight operations paradigms (tactical 
MCC support-centric), due to the lack of capabilities or 
confidence on the integrated vehicle/system health data 
provided by the on-board systems. 

Since acceptable levels of automated health state 
determination for the component and subsystem 
management levels have been achieved (Built-in-Test, etc) 
in existing spacecraft systems, we will focus on the needs at 
the vehicle/system management level. Vehicle Health 
Management at this level requires the integration of 
individual vehicle subsystem health and status data, 
generated at the component or subsystem levels, to obtain or 
formulate an assessment of the vehicle-level health state. In 
the case of the System Health Management, the integration 
must take place for the different elements of the System. 
Therefore, the main functions and capabilities available on- 
board for managing the vehicle/system health are: 

® Correlating all the subsystem data to determine vehicle 
functional availability . 








• Detect and isolate vehicle failures to a “root-cause” or 
reduced ambiguity group amongst all. the anomaly 
indications from the subsystems. 

• Perform vehicle (system) - level impact assessment to 
provide post-failure functionality degradation, 
redundancy degradation, and “critical to” information on 
components that pre- failure had less criticality. 

® Track subsystem performance degradation with vehicle 
(system)-level implications (e.g. negative energy balance, 
cooling deficiencies, air quality or consumable 
degradation). 

• Perform early event detection of anomalies, to identify 
component, subsystem or system performance data that 
generates residual values from expected behavior. The 
real-time detection of such residual data may enable real- 
time or off-line multi-variant analysis that may identify 
the onset of potentially critical failure. 

The information generated by a vehicle-level system health 
management function serves as the foundation of data for 
robust cockpit or space-based control center functions. 
C&W message generation and classification should be 
based on the correlated and integrated set of failure 
indications from multiple subsystems affected by the 
failure. The lack of this capability on current systems 
results in C&W systems that generate cryptic messages 
based on individual failures or anomaly data, with no 
indication of which are “root-cause” (e.g. power switch trip) 
versus just conditions that resulted from the root-cause 
failure (power switch loads losing power). This will reduce 
the number of nuisance-messages and provide the crew and 
operators with actionable failure information on system 
failures that require little extra diagnosis to respond or 
understand. Moreover, C&W messages should have the 
capability to either trigger automated response actions, or 
provide high-confidence recommendations for individual 
actions or procedures to execute in response to the failure. 

In order to provide a complete picture of the failure 
condition and the resulting state of the systems, the 
information related to the nature of the failure must be 
combined with the impact of such failure to the system, 
which we refer to as “impact assessment.” Currently, 
operators and crew determine the impact of system failures 
using precompiled documentation that describes the impact 
of certain significant failures to critical functions, 
components, or their redundancy. It is, therefore, necessary 
for them to perform the complete failure diagnosis 
manually, before they can access and interpret this “impact 
analysis” documentation correctly. In future cockpits or 
space-based control centers, the function" of isolating the 
failure to the lowest possible and reasonable level and 
identifying the impact to the system should be automated or 
autonomous, and integrated with the G&W system. This 


capability will result in the availability of actionable 
information necessary to automate the response or enable 
the crew to promptly understand the actions they must take 
and be fully cognizant on the state of their vehicle or base. 

Large and complex space systems that are to be managed 
primarily by small space-based crews should provide 
automated and autonomous FDIR capabilities at all 
management levels, to ensure crew and system safety and 
the preservation of critical functionality. However, 
automated FDIR actions should only be initiated when 
there is a high level of confidence that the triggering 
conditions are truly met, to prevent further failure 
propagation or fimctionality degradation. Currently, 
automated FDIR is common for subsystem components and 
the subsystem itself, where the conditions can be bounded 
to a limited number of parameters and rules. However, 
vehicle-level FDIR is normally performed manually, since 
the combinatorial aspects of different subsystem failures 
and conditions to respond to are significantly more complex 
than when isolated to a single subsystem. For vehicle-level 
FDIR it is necessary to correlate subsystem data with 
vehicle domain knowledge, such as system hierarchy, 
intersystem dependencies and fault propagation trees, to 
gain the entire failure condition and health picture that may 
lead to triggering multi-system reconfiguration actions. 
Hence, this information must be captured and made 
available on-board for health management applications to 
use for real-time data integration/corxelation that leads to 
the formulation of vehicle (system) -level health state 
assessment. The health management applications must, 
therefore, provide highly deterministic and reliable vehicle- 
level health information that can be relied upon as triggers 
of automated failure responses and multi- system 

reconfiguration sequences in response of failures, or 
dynamic system and mission events. 

Beyond providing data for safety critical functions, health 
management data can be used to enhance the efficiency and 
quality of information provided to the crew via the Crew 
Interface System, thus mprovmg^fe awareness 

and reducing their systems management workload. 
Continuous systems monitoring, currently an MCC 
function, must migrate on-board, and be executed 
autonomously by the spacecraft or space-based systems, 
reporting integrated and “raw” information to the crew and 
MCC (when available) for oversight. 

The data generated to perform the functions already 
discussed, must also be used to execute other Systems and 
Mission Management functions on-board, as stated before. 
Thus, vehicle health and status data must be available in the 
appropriate buses and formatted according to interface 
requirements from mission planning systems. 
Task/procedure automation systems , human computer 
interfaces, and space-based maintenance support 
applications. 
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Task and Procedure Automation 

Whereas today’s operations and system design rely on the 
human to provide the link between the system or vehicle 
data / commands and operational products, such as 
procedures, advances in cockpit automation technology 
offer the potential to break that dependence. Enhanced on- 
board automation capabilities at the vehicle and mission- 
level management may lead to the transition from human- 
centered task / procedure execution to automated or 
autonomous execution with human oversight, retaining 
inhibit and override capabilities. 

There are three main elements which participate in the 
process of selecting and executing a task or procedure in 
space flight. These are the health state of the systems 
necessary for the execution of the procedure, the mission 
timeline (which can drive the execution of procedures 
associated with scheduled activities), and finally, the 
procedure itself. To achieve full autonomy of the execution 
sequence, the executive functions onboard must be able to 
perform a process similar to the one performed by humans 
when executing this activity. This requires determining 
when the procedure has to be executed (timeline driven or 
event/condition driven), verify that the system/mission 
conditions support the execution, execute it, and verify 
successful execution. 

Applications to automate the actual execution of a set of 
actions in a task/procedure, and even to verify the 
individual effect of each command (end-item response), 
have been implemented successfully in robotic space flight. 
Automated command sequence execution is feasible when 
conditions to initiate execution and to verify successful 
execution are simple and straightforward. This is often the 
case for many nominal and off-nominal subsystem level 
tasks, and for a limited amount of vehicle-level tasks. 
However, when the conditions for execution or system 
readiness verification are complex, such as it requires 
vehicle-level health state assessment data, human-input is 
required to initiate and carry out the execution. The human, 
then, must be available to execute and remain fully 
knowledgeable on the conditions and actions. If the only 
humans available are the on-board crew, the burden upon 
them for real-time vehicle / system management is 
significant. Thus, the automated execution capabilities of 
single or sequences of commands (already existing), must 
be coupled with automated systems health management 
capabilities previously described, and also have electronic 
access to the current mission timeline. » 

Ultimately, future spacecraft-systems and space-based 
control centers should have the capability to automate the 
execution of: 

1. Nominal vehicle/system-ievel tasks. These are 
sequences of actions/commands that have a nominal 


planned execution, and must be executed upon pre- 
determined system, mission events or other conditions. E.g . 
execution of tasks to power up dormant systems (multiple 
subsystems), perform systems checks, and upon verification 
of successful completion, perform system powerdown to 
dormant state. 

Under nominal vehicle/system-level task we include the 
sequences necessary to perform system activation/de- 
activation, mission phase/state transitions, system-level self 
test, and routine maintenance sequences (e.g. power system 
battery maintenance charge/discharge cycles). Nominal 
tasks may be initiated upon scheduled events, automated 
systems requests, such as requests from the health 
management system or the automated planner, and human 
requests. 

2. Off-nominal vehicle / system level tasks or the vehicle- 
level Fault Recovery of vehicle-level FDIR. Kg. execution 
of critical system reconfiguration upon the detection of a 
major thermal cooling loop malfunction, which may cause 
further functional degradation if no action is taken. 

The off-nominal sequences, while in most cases pre- 
defined, are executed in response to unexpected system 
conditions caused by system failures that require system- 
level actions, mission events, environment conditions, or 
human-related events. Likewise to the nominal sequences, 
these are triggered by other automated systems, and human 
requests. 

The current operations paradigm divides any automated 
actions from the Operational Data File (ODF) that contain 
the procedures for both nominal and off-nominal 
conditions, relying on the human as the interface. However, 
the enhancement of autonomy capabilities and automated 
functions integration (i.e. health state determination, 
planning / re-planning, and task execution), should drive a 
direct relationship between automated tasks or command 
sequences execution, and readable operational procedures 
that enable the human to perform manual execution of such 
command sequence, if so desired or required. Whereas this 
paradigm shift may not occur in one upgrade of systems 
design (e.g. Space Shuttle to CEV) it should set the aim for 
technology research, application development and system 
design for future cockpit or space-based control center 
implementation. Therefore, such paradigm shift may evolve 
from current modus operandi to enhanced automation of 
system-level task execution and integration of vehicle-level 
autonomy capabilities, while maintaining the traditional * 
ODF products, to merging such procedures with the 
command scripts or other artifacts that are part of the 
automated sequences.. 

Mission Planning and Re-Planning 



Mission planning is currently one of the “long-poles” 
towards automating tactical mission management functions 
on-board. It is particularly complex, due to the amount of 
complex resource and flight dynamics analysis required to 
validate potential plan options, as well as the need to 
integrate such with detailed knowledge on the health state 
of the vehicle. Furthermore, the integration also has to be 
extended to automated task execution components, to 
introduce automation capabilities to the execution of the 
selected mission timeline. Currently, for ISS operations it 
takes a team of 50 mission planners to develop the timeline 
for one crew member. For ISS real-time operations support, 
NASA has been able to reduce the core Flight Control Team 
(FCT) contingent from six to two flight controllers during 
night and weekend hours, where the execution activity is 
low or the crew is in their sleep period. However, the 
planning support, which also represents one of the largest in 
the FCT, is still required. This discipline is also one. in 
which the crew has less capability on-board, and thus 
dependent almost entirely on earth-based capabilities for 
mission plan analysis and real-time re-planning. Whereas 
for strategic and long term planning, including significant 
mission re-planning and mission goal re-prioritization, this 
may continue to be acceptable, the dependency for short 
term and real-time re-plaiming are likely to not be 
sustainable on future exploration missions. Therefore, the 
main requirements discussed in this section revolve around 
automation capabilities necessary to shift the tactical 
planning function from MCC to the cockpit or space-based 
control center. 

Mission Planning can be generally divided into the 
extensive analysis required to develop the plan and 
developing the flight plan timelines. The analysis includes 
mission goals and objectives definition, flight dynamics, 
resources (consumables and otherwise), and crew time. The 
flight plan design includes the integration of system 
resources, with required activities, procedures, and crew 
work cycles. 

The analysis component often requires significant 
computing resources and integration of different types of 
information and expertise. It can also be performed or 
updated in a non-real time basis, and therefore may not be 
required to migrate on-board. However, a subset of such 
analysis capabilities is' required for the tactical element of 
re-planning timelines to accommodate for human, mission 
or system changes. Such is flight dynamics and system 
resources, like power, thermal and propellants. 

The cockpit or space-based control center capability goals 
for the planning/re-plannmg function are therefore, centered 
in the crew’s ability to manage and change the mission plan 
to adapt to the dynamic characteristics of a manned mission. 
Specifically, the combination of on-board automated 
planning applications and the crew should be able to 
perform the following functions with no assistance from 
MCC required in real-time. 


1 . Perform significant or minor mission plan updates, and 
required analysis, to exercise and early return to earth. 

This capability primarily supports abort modes and is based 
in Flight Dynamics analysis or pre-determined template 
selections to execute. This capability is directly related to 
the execution of such mission abort by the spacecraft C&C 
components and other subsystems, such as GN&C. 

2. Enable the crew to develop a short-term flight or 
mission plan, based on their assessment of mission 
goals and objectives and state of the systems and 
mission. This includes the ability to develop “what-if 
scenarios” to optimize their plan development. 

To achieve such capability, the planning applications will 
require analysis capabilities and built-in knowledge on 
activity Ground Rules & Constraints, and procedure 
information, to ensure no conflicts between the developed 
plan and vehicle system or crew resources. This capability 
is also tied to other on-board system data, such as current 
state of resources, and the availability of vehicle/systems 
functions and automated sequences/procedures for specific 
activities. 

3. Automated execution, when determined by humans, of 
the mission or flight plan activities. 

This capability is divided between different Systems and 
Mission Management components, such as: 

-The planning component that controls the timeline and the 
activities, with all the, rules, constraints and procedures 
associated with such activities. This component controls 
manages the specific tasks that need to be performed, and 
the timing of the execution. 

-The Task / command autonomy applications that will 
receive the activities to execute and perform the execution 
of associated command sequences, etc. 

-The System Health Management application that provides 
key information on system availability and functionality of 
the subsystem, which is necessary to validate the activity 
rules and constraints. 

-Crew interface system, which provides the information to 
the crew and allows them to manage the timeline execution, 
including the ability to override, inhibit or initiate such 
execution. This element is particularly important since 
disengaging or negating the control of this function from 
the crew would result in a net loss of management 
capabilities for them, instead of enhanced capabilities for 
mission management. 

All of these components must interface with each other in a 
framework that enables functional autonomy, sharing and 
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providing information and actions to each other and the 
crew. 

These capabilities must result in the crew’s ability to 
understand the plan to execute, modify it if necessary, and 
provide oversight on the automated execution of such (if 
desired) with no or minimal MCC assistance. 

Human-Comp uter Interface 

An effective and efficient Human-Computer Interface (HCI) 
is essential to the crew’s capabilities to understand the 
mission plan, to execute and modify it if necessary, and 
provide oversight on the automated execution of such (if 
desired) with no or minimal MCC assistance. Developing a 
common and effective HCI and applying it to the CEV, the 
Lunar / Mars Surface Access Modules, as well as ground- 
based mission control sites and lunar habitats is cost- 
effective in terms of both equipment and training. A 
common and effective HCI is also critical for shared 
situation awareness and collaboration across a network of 
humans in space and ground-based controllers. 

Recent advances in display and computing technology have 
made feasible the development of a common HCI utilizing 
large format, high resolution virtual displays which allow 
for a more efficient and intuitive presentation of information 
graphically, while maintaining the same “look and feel” 
from one implementation platform to another. Given the 
highly autonomous nature of future long duration human 
space missions, this common HCI should strike an optimum 
balance between keeping the human engaged and “in the 
loop” and total automation control. A common HCI would 
also provide for increased situation awareness, as well 
enhanced safety and mission effectiveness, while reducing 
the training and maintenance costs typically associated with 
complex HCIs interfacing with complex system of systems. 

While we have discussed the requirements for automation 
capabilities to meet future Exploration mission 
requirements, the commercial and military aviation industry 
has shown that increasing automation in the cockpit always 
has unanticipated consequences. Every time a problem is 
solved by technology, a new one will be created. A 
particularly relevant example is what has been termed 
“automation surprises”, or the failure of the human operator 
to track, monitor, or anticipate the actions of an automated 
system leading to unintended system behavior 1 . A better 
understanding of the factors that contribute to these 
automation surprises will allow industry to detect and 
counteract the risks that may arise from implementation of 
new automated systems. Design-induced human error and 
automation problems cause nearly 80% of aviation 
accidents, which attributed to poor interfaces of the 
automation capabilities to the crew.. Additionally, a crew 
interface architecture that is not human-centered is often 
unable to accommodate change and growth. Lessons 


learned in the commercial and military cockpit has shown 
that the best approach to human / automation interface is to 
define the optimal role for the crew first, and make the 
system conform to that role, resulting in increased safety, 
reduced training, greater efficiency, and higher mission 
success probability. One aspect of automated systems 
interfacing to humans is that they often present more 
information than the human can process in the time 
available, resulting in an information overload condition. 
Increasing crew responsibilities, more data to manage, and 
more automation leads to increased training requirements, 
increased workload, and a higher error potential. In the 
absence of a good cockpit information management, 
integration and layout design, the human crew often does 
not recognize potentially important information. 

Traditionally, human space programs from Mercury through 
Shuttle have expected the crews to generate their situation 
awareness from raw data gathered from electromechanical 
gauges and other tactile instruments, such as knobs and 
switches. Even with the advent of feasible CRT and 
subsequently flat-panel LCD technology for space, the 
interface was still designed to mimic a largely heritage 
electromechanical presentation of data augmented by 
rudimentary textual capabilities. Such an interface design 
presents a major challenge to human operators because the 
input bares no resemblance to the cognitive structures it is 
trying to create. One study found that nearly 25 percent of 
commercial aviation pilots who accessed a flight- 
management computer information screen containing 
erroneous data failed to detect the error due to the poor 
layout of the information. Such was the case with the 
original display layout in the Shuttle. The information 
presented on the original Orbiter CRTs was mostly text 
oriented and required astronauts to scroll through pages of 
monochrome textual data to view the information. With the 
implementation of the Multifunction Electronic Display 
System (MEDS) cockpit upgrades, full color graphics, cues 
and warnings indicate the priority of each specific problem 
now presented to the astronauts on the new flat panel LCD 
displays. Unfortunately, despite the modem graphical 
capabilities of MEDS, it was decided to replicate the 
original orbiter elecromechanical layout in the HCI in order 
to keep the panel “familiar” and avoid retraining costs, yet 
neutering the potential for an enhanced and more cognitive 
approach to the presentation of data and information. 
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Figure 3 Shuttle MEDS, while using modern LCD flat 
panel technology still mimics the cognitively challenging 
presentation of elecromechanical gauges and indicators. 

An historical assessment of the Apollo crew interface 
indicates that for missions, the astronauts had to rely on 
significant intuition, mental calculations and estimates to 
identify and navigate landing areas and footprints. 
Additionally, the Apollo astronauts were encumbered with 
the need to memorize a vast number of vehicle specific 
procedures, operational information, switch settings and 
positions that necessitated a lengthy and expensive vehicle 
specific training regimen. Given that the next generation 
human-space exploration mission objectives well exceed 
that of heritage Apollo, a significantly more capable and 
common HCI that can be extended across exploration 
platforms will be required. 

Advances in cognitive psychology, as well as computer and 
display technology since Apollo have made feasible the 
development of a common HCI utilizing large format, high- 
resolution displays which allow presentation of information 
more graphically and more intuitively to the operator while 
maintaining the same “look and feel” from one spacecraft to 
another. Such an HCI would provide many benefits over 
traditional human space crew displays. Large format 
displays together with sensor fusion and computer 
generated information make it possible to create intuitive 
“better than out-the-window” fimctionality. Reconfigurable 
and upgradeable software components could be used to 
support operator decision making and situational awareness, 
while allowing for increased extensibility and a more cost- 
effective system. On demand information and procedures 
could be presented to the operator readily in any desired 
location within a display or displays, providing for better 
information access, better data integration and increased 
situational awareness. Perhaps most importantly, the HCI 
should be multi-purpose; it can be used by an operator 
onboard the CEV, onboard a Mars spacecraft, from a 
remote lunar habitat or from an earth-based mission control 
center without a huge cognitive adjustment. Specifically, 
crew and mission controllers should be presented with 
information allowing them to remain acutely aware of 
vehicle endurance parameters and windows of opportunity 
for critical mission milestones, such as trajectory adjustment 
bums, landing, ascent and rendezvous. HCI capabilities 
must be present to reduce crew / controller cognitive 
workload while enhancing situation awareness. 

Central to the HCI would be visualization tools and 
presentations (e.g:, on-screen smart checklists, advanced 
vision systems and predictive visualizations) with 
capabilities for various selectable orientations such as a 
“God's eye” view, “wing man” view or “out the window” 
view for enhanced situation awareness and mission 
planning. The capability must exist to provide on-demand 
information access and “sharing” of data among spacecraft 
crews and mission controllers on the ground (be it Earth or 


Moon). Finally, the HCI should be easily reconfigurable 
and tailorable to support multiple system configurations 
such as crew positions while retaining its fundamental 
common “look and feel”. 

Developing effective HCI for space is obviously not 
without its challenges and complexities. Several lessons 
have been learned by industry over the past several years to 
reduce design complexity inherent in the development of 
human-centered designs (HCD) for space, commercial and 
military applications. First, effective development teams for 
HCD must be comprised of different disciplines. Essential 
to an effective and well-balanced design team is an 
adequate representation of human factors expertise with 
enough authority to ensure human factors principles are 
considered and implemented in the design. When product 
specifications are written, the product teams take the results 
of the previous human factors studies and analyses and 
construct the necessary specifications. Also, human factors 
personnel should be extensively involved in the testing 
process along with the customer, users, certification 
authorities, and product groups. Flight testing should ensure 
that the human factors requirements are accomplished. 
These lessons learned may be encapsulated within an 
effective HCD process. 

5. Enabling Technology Development For 
the Smart Cockpit 

Understanding the capability requirements for future 
cockpit or space-based control centers is necessary to 
evaluate technology or applications that may enable such 
capabilities. This paper provided an overview of such 
capability requirements for specific Systems and Mission 
Management functions. This section offers an overview of 
some technologies that show potential or currently offer 
solutions to meet such capability requirements. Whereas we 
do not cover the entire spectrum for spacecraft functions, or 
technologies available for one specific function, the 
examples areas selected offer an overview of the technology 
characteristics that ought to be considered* and development 
to transition them into on-board applications. 

Integrated System Health Management (ISHM) 

The term Integrated Vehicle Health Management (FVHM), 
ISHM, or even Integrated Health Management (fflM) has 
been used to describe multiple, distinctly different 
technology or applications in the industrial, aerospace and 
military sectors. The same acronym has been used to 
describe smart sensors that perform local processing of the 
health data acquired on a single system, to inference engines 
that correlate massive amounts .of data from multiple 
distributed systems to provide an integrated system health 
view. Furthermore, different technology approaches to 
achieve a similar solution are also labeled with similar 
acronyms. In this paper we are focusing on technology and 
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applications that provide solutions to the problem domain 
described in previous sections; the integration of subsystem 
or system element individual health state information to 
provide a vehicle or system- level health state. 

To perform such function, it is necessary to go beyond 
traditional rule-based methodologies, since the complexity 
of system relationships and vehicle-level fault propagation 
exceeds the ability of such methodologies to capture enough 
of the domain space to enable true health state integration. 
We look beyond the traditional algorithms, towards 
applications that are able to capture inter-system 
dependencies, system hierarchy, system-level . fault 
propagation, and have the ability to utilize this information 
for real-time correlation of individual health and status 
inputs to achieve the vehicle or system level diagnosis and 
health state assessment. Yet, the determination of vehicle- 
level health state leads to system and mission critical 
decisions, such C&W and FDIR, thus the methodology to 
obtain such must be folly deterministic and reliably 
accurate. Therefore, the candidate technologies for this 
function must combine the advance reasoning capabilities, 
with reliability and determinism. 

Model-based system health diagnosis has been utilized 
widely in different industries to capture system information 
described above, and utilize such for real-time correlation 
and reasoning. Within model-based approaches there are 
several options, based on what type of model is utilized. 
These options include: 

Model-based state estimation , such as the Ames Research 
Center-developed Livingstone software, which uses a model 
of the system to predict its behavior. If actual behavior 
diverges from the model's predictions, a diagnosis is made 
to isolate the cause of the discrepancy to a specific failure. 
Therefore, ultimately it is a comparison of expected 
behavior vs. actual behavior that triggers the diagnosis. 
Livingstone has been used as the reasoning engine for 
multiple prototypes of vehicle-level health state diagnostic 
engines, including flight test on Deep Space 1 and EO-1, 
where it was used as part of a flight experiment. 

An alternate solution is model-based methodologies based 
on multi-signal dependency modeling and evidence- 
physics-based run-time reasoning. On this approach, the . 
model represents the sub-system or system element 
dependencies, capturing components, functions, 
propagation paths, sensors, tests, probabilities and other 
information. Diagnosis is accomplished by following 
dependencies from the tests or sensors, which are the 
indications of failure symptoms, back along known 
propagation pathways, to the originating cause of the 
failure, such as a component or a particular failure mode of 
a component. The multi-signal dependency models capture 
the vehicle’s topology, functions, intersystem dependencies 
and fault propagation paths across multiple subsystems (e.g. 
power system failure resulting in the loss of power to 


multiple components). In the system view, these are inter- 
element dependencies (e.g. Orbiting element to Planetary 
Base element). 

This approach is typically comprised of two application 
components; the tools used to create the models and 
perform analysis on the system design as represented in the 
models, and the run-time reasoners that perform the 
diagnostic analysis in real-time. The off-line models are 
transformed into Dependency Matrices, or other executable 
code that can be reasoned upon efficiently. The off-line 
models can be built in the form of graphical representation 
of the dependencies, data-bases, etc. Whereas this approach 
has not utilized for system health management applications 
in manned space systems, it has widely been used in the 
commercial and military aerospace industry for different 
off-line and real-time integrated diagnostic applications, as 
well as in other sectors such as the automotive and 
petrochemical industries. Qualtech Systems Inc. (QSI) 
TEAMS tool set combines the off-line diagnostic modeling 
and run-time reasoning application for an integrated 
solution. In QSI’s solution, the domain knowledge capture 
is enabled by Team-Designer, which provides graphical 
modeling, and testability analysis, as illustrated in figure 4. 
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Figure 4 TEAMS models provide system design diagnostic 
analysis and executable domain data for diagnostics 


This off-line Windows-based tool provides capabilities to 
capture vehicle and system dependency and topology 
information, as well as many other parameters from 
component reliability to Mean-Time-Between-Failures 
(MTBF) data, if desired. This information is then exported 
as a Dependency Matrix for use by the TEAMS-RT 
diagnostic reasoner, which correlates the individual results 
of the test points, which can be populated by sensor inputs, 
algorithms, etc, to provide the integrated view of the health 
state, or fault propagation path. DSI International, Ina 
provides the eXpress tool, which also targets the domain 
capture and diagnostic analysis capabilities, to ensure that 
the vehicle design supports the required levels of fault 
coverage, health visibility into all aspects of the system- and 
ultimately, the optimum instrumentation coverage and 
placement. Figure 5 provides an example of and eXpress 
diagnostic model. 
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Figure 5 eXpress diagnostic models of a manned 
spacecraft Reaction Control System design 


Whereas the eXpress models are used for analysis off-line, 
these models can be exported to other run-time model-based 
diagnostic reasoners, such as the Impact Technologies 5 
ReasonPro, or custom develop reasoners that can read the 
standard model-export format. Through the early use of 
tools like eXpress, the platform design is optimized for 
Fault Detection and Isolation capabilities and well as sensor 
selection and placement to support the automated health 
management application design. Prognostics / physics of 
failure candidates are also selected during the front end 
diagnostics design using eXpress. Effective Run-Time 
health management is based on the run-time application’s 
interface with the front end design process, which in turn 
provides the domain information necessary for effective 
correlation and integrated reasoning. The developers of 
eXpress, for example, have teamed with Impact 
Technologies, developer of ReasonPro, to provide an 
integrated solution, dubbed eXpress-RT, of off-line and 
run-time solution, similar to the QSFs TEAMS toolset. 

Ultimately, these COTS applications do not provide, 
standalone, the entire solution to meet specific ISHM 
capabilities required for future cockpits or space-based 
control centers. The specific requirements for vehicle and 
mission critical functions, including systems and mission 
management automation, will lead to custom designed 
applications for the operation systems. However, the 
current proven capabilities both in diagnostic and testability 
analysis and run-time applications, demonstrate that the 
technology foundation necessary for the custom 
applications has been developed. Furthermore, the 
successful heritage and maturity of such technology, with 
demonstrated detenninistic behavior, is a significant risk 
reduction over the implementation non-aeterministic or 
lower Technology Readiness Level (TRL) rated technology 
approaches to achieve similar automatedneasoning 
capabilities. This risk reduction is especially significant 
since any such cockpit-based application must endure the 
stringent Validation and Verification (V&V) process, as 
well as flight certification required for human-rated 
systems. 


The combination of such technology (multi-signal 
dependency models with evidence/physics-based 
reasoners), with traditional approaches (i.e. rule-based), and 
system-specific interfaces and applications, will lead to the 
development of applications that provide the required on- 
board ISHM capabilities. Moreover, the customization of 
models, reasoners, or the combination of capabilities with 
other non-deterministic approaches providing supporting 
information, may lead to end-to-end health management 
capabilities that cover the real-time critical requirements, 
and also maintenance and long-term system prognostics 
needs on-board. 

Command Scripting and Class V lETMs 

Whereas, integrated health management is necessary to 
enable any automation of other system or mission 
management functions, the automated execution of tasks or 
command sequences is the “execution aim” of the 
automation systems. To achieve significant on-board 
independence from MCC to perform systems and mission 
management, it is necessary to not only provide the crew 
with accurate and timely information on the integrated 
health state of the system, but also to provide them with 
automation capabilities for nominal tasks and off-nominal 
event response. Autonomy capabilities for task automation 
can be embedded in different components of the system. 
Health management applications have the insight into 
vehicle health state and provide direct information related to 
conditions that may required automated vehicle-level 
responses, thus it could perform the vehicle-level FDIR 
functions. The planning systems manage the mission 
timeline, and therefore could have an execution branch to 
automate the execution of tasks in the timeline. 

Nonetheless, comprehensive task automation must cover all 
the task execution options within the cockpit or the control 
center, including vehicle-level FDIR, nominal tasks in the 
timeline, and crew or operator directed initiation of 
automated execution. Furthermore, it must not be exclusive 
to one specific functions, such as health management, but 
aide the crew in any nominal or off-nominal system 
management task. Hence, we focus on technologies that 
may provide such comprehensive automation capabilities 
for task execution. 

Hie most basic mode of automating the execution of 
commands is to provide command scripting capabilities on- 
board, such as pre-detennined command sequences can be 
executed upon pre-determined triggers. The triggers can 
range from human input to system health management 
inputs, planning system inputs, etc. The level of autonomy 
is then decided by the capabilities that trigger the sequence 
execution initiation, which can be highly complex 
algorithms, or simple rules, such as parameter limit 
violations, or modes/states changes. 
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There are currently command scripting applications that 
have been human-rated and are currently in use in a variety 
of aerospace and industrial applications. The Draper 
Laboratories Timeliner application is used in ISS payload 
operations to automate different aspects of ISS Rack and 
payload systems initiation, operations and safing. Timeliner 
is also resident in the C&C flight software, however it has 
not yet been extensively used for core-system task 
automation. 

ICS’ Spacecraft Command Language (SCL) also provide 
command scripting capabilities, similar to those of 
Timeliner, which can be integrated as a stand-alone 
command scripting application, or part of the C&C system 
for vehicle management. 

As command automation technology implementation takes a 
foothold on future spacecraft design and operations, the 
ideal progression of capabilities point beyond the execution 
of pre-defined static command sequences. The static 
sequence execution upon dynamic conditions should shift 
towards the dynamic command sequences, not entirely pre- 
defined, upon dynamic conditions. Moreover, these 
capabilities should merge the traditional crew and operator 
procedures for nominal, malfunction, activation and 
checkout, etc, resident in the ODF, into an electronic 
repository of tasks that can be executed manually, with 
mixed initiative (autonomous) or fully automated. 

Class V Electronic Technical Manuals (lETMs) reflect such 
capabilities, where the procedure action are determined by 
the dynamic system or mission conditions, instead of 
having a static sequences pre-determined prior to flight. 
There are several approaches to Class V IETM 
development. One of those is offered by QSI, utilizing the 
same diagnostic models developed for forth the testability 
analysis, and the run-time integrated diagnostics and health 
assessment. This commercial product, called TEAMATE, 
dynamically produces operator actions to further diagnose 
health anomaly ambiguities to achieve a root-cause 
diagnosis, as illustrated in Figure 6. 



Figure 6 TEAMATE user interface for dynamic procedure 
execution , based on enhanced diagnostic models 

The applications and capabilities discussed above provide 
the foundation for future applications, perhaps combining 


different technologies and approaches for decision 
autonomy and execution, that will provide critical 
automation capabilities for command and activity execution. 

Constraint-Based Planning and Scheduling 

As previously discussed, current short-term timeline 
development is a highly manual task, as well of the real- 
time re-planning and modification of such. Large 
contingents of people from different system disciplines (e.g. 
planners, resource analysts, systems operators, etc) are 
necessary to maintain the plan current and re-plan in real- 
time. While there is automated planning technology that can 
address both mission plan development pre-flight, and real- 
time re-planning, we are focusing on the latter due to its 
tactical nature, which has more significance on the 
capability requirements for mission management 
automation on-board. 

Even when the mission plan has been developed, and 
different timeline options for short-term execution are 
available, certain amount of real-time analysis is required to 
validate the current timeline, or develop additional options. 
Moreover, when the system state or mission conditions 
change (e.g. crew member ill) the new asset availability, 
both crew and systems, changes the validity of the current 
plan, and therefore must be re-planned. Therefore, the 
automation capabilities must cover limited systems, flight 
dynamics and resource analysis, as well as constrain 
analysis, de-conflicting techniques, and plan options 
evaluation and rating capabilities, as illustrated in Figure 7. 
Automating such planning functions would provide the 
crew with the necessary information to select plan options 
and execution approaches. 



Figure 7 Automation in planning requires analysis and 
plan validation capabilities , as well as close integration 
with the crew and other systems management components 


We, again, revert to technologies with proven capabilities in 
relevant fields, and with similar or superior parameter and 
condition complexity. For this section we focus on 
methodologies that offer model-based mission activity and 
resource representation, incorporating mission ground rules, 
constraints and procedures that is compatible with proven 
automated planning tools. Constraint-based planning 
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provides decision technology that can solve and optimize 
the world’s most complex scheduling and resource 
allocation problems. 

The main purpose of automated planning capabilities is to 
enable the crew to control their mission timeline and have 
the capability to change it if required. Constraint-based 
planning enables strong mixed-initiative automation, 
integrating the human input, or override into the process. 
Constraint-based planning technology also relies on domain 
and scheduling models, which contain the relevant activity, 
constrain, and ground rules information to be applied to the 
re-plan activity. Such models can be developed in a variety 
of ways, from more automated to manually, as in current 
processes. Hence, automation in the development of such 
models could further increase the efficiency of the process. 
However, automation of domain model development and 
pre-flight mission plan development are not necessary to 
migrate tactical mission re-planning on-board, since even 
complex missions to Mars will support some level of 
communication with Earth, thus retaining the ability of 
acquiring additional or modified models, as well as long- 
term mission plan files. 

The heritage of such algorithms, enabling the automation of 
complex scheduling problems in the aerospace (and 
petrochemical) industries also ensures the ability of such 
approaches to he implemented on systems with strict V&V 
requirements. 

Information Management and Displays 

Since NASA’s first rendezvous attempts during Gemini, 
culminating with the Lunar landing and rejoin of the Apollo 
Lunar and S ervice Modules, one of the key issues for a safe 
completion of human spaceflight mission objectives has 
been the ability of the crew to have adequate visual 
references. This capability has been essential to both 
manually controlling the spacecraft during 
rendezvous/ docking & approach/landing, and to visually 
verify that automated GN&C involved with these operations 
is working properly. This capability has always been in 
direct conflict with the structural engineering desire to 
eliminate or at least to minimize the spacecraft windows 
providing the outside view. Even more critical to ground ' 
mission controllers and planners is the ability obtain and 
maintain a high degree of situation awareness involving the 
vehicle and its mission at a distance without the benefit of 
natural views through windows of any kind. 

Advances in avionics computing, graphics processing and 
display and sensor technology, since Apollo, makes it 
possible now to present a “better than natural”’ view outside 
the spacecraft. Using Advanced Vision Systems, next 
generation human space exploration vehicles could be 
equipped with an exterior array of im age plane sensors and 
a high-resolution landing zone terrain database that would 


be used to render a synthetic image of a virtual outside 
world. The computer generated scenery would be 
appropriately overlaid with real images fused from a 
combination of other sensors including visible, infrared 
(IR), millimeter wave (MMW) and LIDAR to name but a 
few. Such a system (display shown in Figure 9) may 
actually be superior to natural human vision in that it could 
allow crews, operators and controllers to “see” in darkness, 
bright sunlight conditions or through Martian dust storms. 
Moreover, the scene potentially could be zoomed and 
manipulated electronically; this would provide operators 
and crew with supra-normal artificial visual acuity, thereby 
affording earlier landing site evaluation, obstacle 
identification and touchdown point redesignation. Fusing 
real world “enhanced” vision imageiy avoids the obvious 
limitations of purely model based, exclusively “synthetic” 
vision systems. In addition to enhancements provided by 
sensors and synthetically generated terrain, flight data 
symbology, data graphics and other vehicle parameters 
would be overlaid in what is called Advanced Symbology to 
provide crew and operators with a single point of reference 
and thereby increase situation awareness while reducing 
cognitive workload. 



Figure 9 . Advanced Vision System Display integrating 
Enhanced Vision, Synthetic Vision and Advanced 
Symbology . 


An historical assessment of Apollo crew interface indicates 
that for lunar landing missions, ft le Apollo astronauts had to 
rely on memory, significant intuition, mental calculations 
and estimates to idenffiy aridnavipB HSiSg areas 
and landing footprint areas (NASA, 1975). Due to the 
limitations of window size and geometry on the LM, it was 
difficult for the astronauts to sense relative size of obstacles, 
descent rate and radial velocity; only a limited number of 
landing site re-designations were available due to these 
constraints (McCandless, McCann, & Hilty, 2003). 
Operators and crew should not have to make command 
decisions through such error prone methods, (Lintem, Waite, 
& Tallent, 1999). Instead of relying on mexnoiy which 
under significant pressure can be flawed at best (Wickens & 
Hollands, 2000), humans engaged in supervisory control 
should He provided with direct perception-action visual 
representations in as natural a depiction as possible. Such a 
direct depiction allows operators and crew the ability to 
establish and maintain a high degree of situation awareness 
facilitating the ability to make correct and timely decisions, 
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and reduce errors in the process (Rasmussen, 1998; 
Shneiderman, 1998). Given the future human space 
exploration parameters of light / dark, polar or equatorial 
landing site requirements of lunar and Martian missions that 
go beyond Apollo, some sort of AVS capability seems to be 
essential for safety and mission success. 

Decades of experience in the military and civilian aviation 
domains has demonstrated that the largest fraction of 
vehicle control problems (e.g., near misses, loss of control, 
controlled flight into terrain) have been due to the inability 
to recognize (due to night , and/or marginal visibility) normal 
objects that the operators normally utilized in maintaining 
situation awareness (SA) and control. This increased the 
pilots’ workload further exaggerating the negative effects. 

Advance Vision Systems (AVS) technology has' been 
demonstrated to significantly increase safety and 
operational flexibility for spacecraft, aircraft and land 
vehicles operating in low or no-visibility conditions. The 
all-condition visibility provided by AVS facilitates 
maneuvering allowing normal operations under conditions 
that normally would dramatically slow or even halt activity. 
AVS affords an opportunity for future windowless cockpits 
for next generation space vehicles. 

The Advanced Vision System consists of three primary 
components: 

® Synthetic Vision - derived from a terrain database; 

® Enhanced Vision - from onboard sensors: video, 
IR, millimeter wave (MMW) radar, or LIDAR 

Advanced Symbology - a visual vocabulary and format 
engineered to make the fused information immediately 
intelligible to the user and indicative of the precise 
relationship of the aircraft ./ vehicle to the environment. 



Figure 8. Large Format AVS display depicting flyover of 
Martian terrain. 


Synthetic vision employs computer-generated terrain 
imagery from on-board databases, and precise position arid 
navigational information to create a presentation of the 
outside world (Figure 2). The presentation to the operator is 
available in 3D perspective, planview, or vertical profile 


foimat dependent upon operator task. Synthetic vision 
eliminates poor visibility (be it because of darkness, or 
small or no windows) as a safety factor and enhances the 
operational capabilities of the vehicle. Synthetic vision may 
be presented to the user in either 2D or 3D and even 4D, 
where time may be manipulated for training and proficiency 
purposes, an important consideration for long-duration 
missions. 

Enhanced vision already demonstrated and certified by the 
FAA for commercial aviation use in marginal visibility 
conditions, (e.g., night “black hole” approaches) using 
infrared augmentation, offers excellent night visibility, 
improving maneuvering accuracy and situation awareness. 
The future addition of millimeter wave radar capability 
promises even higher image quality and obscuration 
penetration. Honeywell has approached the difficult 
problem of fusion by utilizing efficient feature extraction 
techniques from the sensor data that significantly enhances 
the registration of the sensor image with the synthetic one. 
In the case of the use of a head-up display (HUD) to present 
the AVS in the aircraft world, it also enhances the 
registration of the sensor image with the real out-the- 
window view. 

Advanced Symbology employs a visual vocabulary and 
display format that integrates infonnation from all available 
sources, to present what the operator needs in ways that 
require less work to understand and process, improve pilot 
tracking performance and increase situation awareness for 
crews and controllers alike. This display methodology 
would simplify vehicle control by replicating cues basic to 
visual operation — including representation of outside 
relationships— and integrating these cues with navigation 
and directional information like a path vector. 

By exploiting immediately understood relationships 
(affordances) and cues. Advanced Symbology establishes 
temporal and spatial relationships for the crew without the 
need for conscious decision. For instance, advanced 
symbology replicates, the Texture, perspective and color cues 
used to judge movement and distance visually— terrain 
shading, texturing and perspective lines, and the expanded 
field-of-view critical for a sense of directional movement. 

A demonstrated ability to seamlessly integrate the multiple 
sources of infonnation from onboard sensors and databases 
(e.g., navigational, geopolitical, obstacle and charts) is 
critical to the ability to provide the best user interface for 
current and future crew and operational needs. 

Of course, display technology is essential to the realization 
of the application of AVS to space. Large- format and high 
aspect ratio Active matrix Liquid Qystal Display (AML * 
LCD) have recently been used to provide the necessary 
space to merge the rendered terrain and advanced 
symbology in a way that enhances the operator’s 
performance in path and energy management and greatly 
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improved situation awareness. A large, landscaped display 
can provide several operational advantages to the 
workstation as compared to a square porthole into which the 
same information must be squeezed to fit. In our human 
factors evaluations of the large landscaped displays, we 
have observed the potential to reduce task completion time, 
reduce input errors, increase situation awareness and 
produce an overall reduction in crew workload. 

Active matrix Organic Light Emitting Diode (AM OLED) is 
an emerging display technology with a significant potential 
to challenge the well-entrenched LCD technology in many 
of its current applications and has attractive qualities for 
space applications. Compared to LCD, the advantages of 
AM OLED include lower power, lighter weight, being 
thinner and all solid state / more rugged, superior image 
quality with wide viewing angle and fast response time, and 
lower cost. AM OLED does not require a backlight or color 
filters like a backlit AM LCD. Currently, glass substrate 
based AM OLED is being developed rapidly with initial 
products such as displays for mobile phones and digital 
cameras already on the market. 

Flexible AM OLED displays fabricated using plastic 
substrates have a potential for being very thin, light weight, 
and highly rugged, with greatly minimized propensity for 
breakage, and lower cost. The current high level of interest 
in flexible displays is facilitating the development of the 
required enabling technologies, which include development 
of plastic substrates, low temperature active matrix 
backplane fabrication technologies, and display packaging. 
Development of flexible AM-OLED display technology 
may be viewed as a natural evolution of the rigid glass 
substrate based AM-OLED development. In actuality, 
flexible AM OLED development represents a paradigm 
shift. Flexible AM OLED displays can provide many 
unique display applications including suit-based and surface 
EVA due to its unique form factors of conformability and 
roll-ability during use, transportation and storage. For 
example, crews exploring the lunar or Martian surface could 
remove a rolled AM OLED display from storage in their 
packs and unroll the display into a large format AVS 
representation of navigational and other mission data and 
then when finished return the display to their equipment 
packs in its space-saving rolled tubular form. 

Commercial industry has been actively involved in the 
development of flexible AM OLED displays since 2000. 
Under a recent DARPA funded program, the world’s first 
AM OLED developed on a flexible plastic substrate was 
demonstrated. This demonstration constituted development 
of low temperature (150°C) a-Si TFT process for 
fabricating the required active matrix circuitry on flexible 
polyethylene naphthalate (PEN) plastic substrates, design of 
the active matrix OLED pixel circuits and display drive 
electronics, and fabrication of a 64x64 pixel test displays 
with an 80 dpi (dots per inch) monochrome resolution. 
Recent demonstrations included 160xlo0(x3) pixel test 


displays (~ 2x2-inch size) with 80 cgpi (color groups per 
inch) equivalent resolution (Sarma et al, 2003). 

The most critical area for efficient use of control station 
space is directly in front of the operator and about 15 
degrees down from horizontal, where resting vision tends to 
fall. Within this forward panel area, using larger landscaped 
displays can minimize the amount of space consumed by 
bezels with respect to the amount of space provided for the 
usable, visible display area. Further, large, landscape 
displays tend to conform to the rectangular shape of most 
forward panels in aircraft and the wide format of the human 
visual field 

In a full screen format, map and terrain graphics on a large 
landscape display can provide unsurpassed situation 
awareness capabilities (Figure 10). For example, in the 
aviation domain, air traffic controller workstations have 
always been designed with the recognition that large 
formats are required for traffic and weather situational 
awareness; the future collaborative decision-making 
environments will make it desirable to provide that same 
level of awareness to the crewmembers and other operators 
via large formats. 

Honeywell has recently completed a series of human factors 
focused flight test programs testing the aviation version of 
AVS. The results indicate a very strong user acceptance 
and usability of the AVS concept. Study participants 
reported that the synthetic terrain was very visible and 
useable in all lighting conditions and the advanced 
symbology was a very useful aid to pilot performance and 
workload. 



Figure 4 4 , AVS concept integrating information 

into a perspective tiled display . 

A potential solution for vision in daylight conditions is to 
put bubble canopies or large windows on the spacecraft. 
However, cost, weight, and structural issues will usually 
eliminate most any solution that places a large picture 
window in any space vehicle. Regardless, missions 
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requiring operations in darkness or in obscuration 
phenomena would still make unaided vision unacceptable. 
Finally, every sensor has a limitation. Therefore a good 
fusion of synthetic and sensor data not only provides a wide 
range of data to select from, but it also allows for the 
optimal set of components to each accommodate a much 
larger variety of conditions and system states (e.g., a sensor 
failure). 

6. Conclusion 

Future cockpit and space-based control centers must employ 
automation capabilities in key areas of Systems and Mission 
Management functions, to successfully meet Space 
Exploration goals and objectives. These capabilities must 
provide the crew with Integrated Vehicle and System health 
information, assuming the function of constant system 
monitoring and integrated performance assessment. In case 
of anomalies, the system must detect and integrate all 
individual system failure indications, and provide the crew 
with C&W information that points to the root-cause and 
provides system impact information such as functional and 
redundancy degradation caused by the failure. 

Automation capabilities should also be implemented to 
nominal and off-nominal task execution functions, to relieve 
the crew from routine systems management tasks, and 
complex diagnosis and execution of time-critical failure 
response actions, while preserving their ability to inhibit, 
override or otherwise control all automated or autonomous 
functions. In the Mission Management category, the real- 
time planning functions must be shifted on-board for 
efficient and timely mission execution, providing the crew 
with sufficient information to alter, or re-plan their short- 
term timeline in response to vehicle, system, mission or 
personal events. 

The level and efficiency of the integration of the crew with 
these capabilities will determine the feasibility of the 
implementation of advance automation capabilities. 
Therefore, the design of an optimal HCI that fulfills the 
cognitive needs, and enables the crew to remain in control 
of all Systems and Mission Management functions is 
tantamount to success. 

As discussed in this paper, there are currently technology 
and applications that can serve as the technology foundation 
for the development and implementation of such automation 
capabilities on-board future systems, while maintaining 
reasonable development cost and preserving or enhancing 
system and crew safety. However, active adaptation, 
development and maturation is required to ensure that the 
readiness levels are ideal when the capabilities are required. 
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